Learning Distance Functions: Algorithms and Applications

This thesis presents research in the field of distance learning. Distance functions are extensively used in various application domains and also serve as an important building block in many types of algorithms. Despite their abundance, until recently only canonical distance functions such as the Euclidean distance have been used, or alternatively various application specific distance functions have been suggested, which in most cases were handdesigned to incorporate domain specific knowledge. In the last several years there has been a growing body of work on algorithms for learning distance functions. A considerable amount of different distance learning algorithms have been suggested, most of which aim at learning a restricted form of distance functions called Mahalanobis metrics. In this thesis I will present three novel distance learning algorithms: 1. Relevant Component Analysis (RCA) An algorithm for learning a Mahalanobis metric using positive equivalence constraints. 2. DistBoost A boosting based algorithm which can learn highly non-linear distance functions using equivalence constraints. 3. KernelBoost A variant of the DistBoost algorithm which learns Kernel functions, which can be used in any kernel-based classifier. I will then describe their applications to various data domains, which include clustering, image-retrieval, computational immunology, auditory data analysis and kernel-based classification. In all of these application domains, significant improvement is made when using a learned distance function instead of a standard off-the-shelf distance function. These results demonstrate the importance of this growing research field. The first two chapters of this work present a general introduction to the field of distance functions, and distance function learning, with some additional background on semi-supervised learning: Chapter 1 Introduction: In Chapter 1 we provide a general introduction to distance functions, and some reasons why the distance learning problem is an important and interesting learning scenario. We then provide a detailed overview of canonical and hand-designed distance functions. The algorithms presented in this thesis are all from the field of semi-supervised learning. We therefore present a short introduction to the field of semi-supervised learning, with a specific focus on learning using equivalence constraints, which is